1. Field of the Invention
This invention relates to video signal processing, for example, processing in which compressed data representing two or more video signals are combined. Embodiments of the invention concern video signals compressed according to the known MPEG-2 standard, but the invention is not limited to such signals.
2. Description of the Prior Art
It is often desirable to mix, wipe or superimpose two or more video signals. For example, a so-called wipe effect might be used to transition between two different scenes in a television programme, or a so-called logo or other computer-generated signal such as a subtitle or a set of credits might need to be superimposed over a video image without otherwise disrupting the underlying image.
With analogue video signals, or even with uncompressed digital video signals, this operation is relatively straightforward. A key signal can be used to control the level of each of the constituent video signals (say, signals xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d) at each pixel position, with the two level-controlled signals then being added together. A basic relationship between the level of the key K signal, the levels A and B of the input pixels and the level of out output pixel at each pixel position might be:
Output pixel value=A(1xe2x88x92K)+BK
This process is carried out for each output pixel. So, if signal A is to be replaced in its entirety by signal B at a particular pixel position, the key signal would be 1 (otherwise expressed as 100%), and if there is to be a 50:50 mix of the two pixels the key value would be 0.5 or 50%.
The situation is much more difficult when either or both inputs is a compressed video stream. In a compressed video stream such as an MPEG-2 video stream, pixels are generally compressed as blocks known as macroblocks, so that it is not possible to derive the value of a particular pixel directly from the compressed video signal.
Compressed video signals are also often subject to an overall limit on the quantity of data that can be used to transmit or store the signal. While there can be some variation from picture to picture, or even from group-of-pictures (GOP) to (GOP), the time-averaged data rate is often constrained to the capacity of a transmission or storage channel. This allowable variation from picture to picture of GOP to GOP can mean that two signals to be combined can have the same nominal data rate but very different instantaneous data rates. So, when constructing a composite video signal from a group of video signals including one or more compressed signals, great care is needed to avoid a data overflow or underflow.
A third feature of compressed video signals relevant to this discussion is that they often make use of motion vectors to indicate blocks of temporally preceding or following pictures which are similar to a block of a current picture and so can cut down the amount of data needed to encode the current picture.
One way of handling these problems is to decompress the entire compressed input signals, carry out the mixing or similar process in the non-compressed domain and then recompress the resulting composite picture.
In systems such as the MPEG-2 system each generation of compression tends to reduce the quality of the resulting images. It is undesirable if the simple addition of logo or similar information causes a deterioration in the overall image quality of the pictures to which the logo information is added.
According to one aspect of the present invention, there is provided a video signal processor comprising a first input arrangement for receiving a decompressed first video signal representing first video information and preserved transcoding parameters associated with the pictures of the signal and relating to the compression,
a second input for receiving a second video signal representing second video information to be combined with the said first video information,
a combiner for combining the second video information with the first video information to produce a picture of the combined information, comprising at least one part derived only from the first signal and at least one other part derived at least partly from the second signal, and an encoder for compression encoding the said picture to produce a combined signal representing the combined information, which encoder reuses at least one of the said preserved parameters for encoding at least the said at least one part of the combined information which is derived only from the first signal.
Preferably, the processor has a first input for receiving a compressed first video signal, and a decoder for decoding the compressed first video signal whilst preserving the transcoding parameters in association with the pictures.
According to one aspect of the present invention there is provided a method of processing a video signal comprising the steps of:
receiving a compressed first video signal representing first video information including transcoding parameters associated with the frames of the signal and relating to the compression,
decoding the compressed first video signal whilst preserving the parameters in association with the frames,
receiving a second video signal representing second video information to be combined with the said first video information,
combining the second video information with the first video information to produce a picture of the combined information, comprising at least one part derived only from the first signal and at least one other part derived at least partly from the second signal, and
encoding the said picture to produce a combined signal representing the combined information, which encoder reuses at least one of the said preserved parameters for encoding at least the said at least one part of the combined information which is derived only from the first signal.
Reusing the preserved motion vectors helps to maintain the overall picture quality because recalculating motion vectors may introduce errors. In addition the picture quality of image parts derived solely from the first signal is maintained.
According to a further aspect of the present invention, there is provided a computer program product providing computer executable instructions, which when loaded on to a programmable signal processor causes the processor to implement the method of said another aspect of the invention.
The invention allows any reduction in image quality due to the recoding of the combined video information to be minimised or at least reduced by re-using preserved parameters for parts of the combined image which are derived from the first video signal.
In an embodiment, the first signal comprises intra-encoded pictures and inter-encoded pictures, and the encoder re-encodes intra-encoded pictures of the first signal as intra-encoded pictures of the combined signal and re-encodes inter-encoded pictures of the first signal as inter-encoded pictures of the combined signal. Intra-encoded pictures are containing the whole video information of a picture encoded according to the coding scheme applied to the picture. An intra-encoded picture does not require information from any other picture when decoded to reproduce the original whole picture. An inter-encoded picture requires information from another picture, e.g. an intra-encoded picture, to reproduce the original whole picture when decoded. An inter-encoded picture is for example the difference between a picture and a prediction of the picture.
In MPEG, an I-frame is an intra-encoded picture, and P and B frames are versions of inter-encoded pictures.
Preferably in embodiments for processing video information including I, P and B pictures, the encoder is arranged to encode I, P and B pictures of the first video signal as I, P and B pictures respectively of the combined signal. Thus the type of picture (intra, or inter; I, P or B) is unchanged contributing to preservation of image quality and reducing the amount of re-encoding.
In the embodiment, the encoder preferably reuses at least one of the said preserved parameters for encoding at least the said at least one part of the combined information of an intra-encode picture which is derived only from the first signal and re-encodes at least one transcoding parameter of at least one part of the combined information of an inter-encoded picture which is derived at least partly from the second signal. In embodiments of the invention, intra (I) encoded pictures are re-coded using preserved parameters for at least parts thereof derived solely for the first signal to maintain image quality. In one embodiment the encoder re-uses the said at least one preserved parameter associated with an I-picture of the first signal when encoding the whole of the corresponding I-picture of the combined signal. The said one parameter is most preferably the quantisation parameter. Thus intra-encoded pictures which are most significant for determining picture quality re-use transcoding parameters to maintain picture quality.
In another embodiment, the encoder re-uses the said at least one preserved parameter associated with an I-picture of the first signal when encoding part(s) of the corresponding I-picture of the combined signal which are not derived from the second signal and re-encodes the said at least one preserved parameter of other part(s) of the corresponding I-picture of the combined signal which are derived from the second signal. Preferably, the said at least one preserved parameter is a quantisation parameter, and the encoder responds to a threshold level of an image parameter, and: reuses the preserved parameter for those other part(s) of the corresponding I-picture of the combined signal which are derived from the second signal and for which the image parameter is one of lesser and greater than the threshold level; and re-calculates the quantisation parameter for those other part(s) of the corresponding I-picture of the combined signal which are derived from the second signal and for which the image parameter is the other of lesser and greater than the threshold level.
Inter-encoded pictures, (e.g. P pictures) are less important to image quality than I pictures. However in a preferred embodiment the encoder reuses at least one of the preserved parameters associated with a P picture when re-encoding part(s) of the P pictures which are not derived from the second signal. Preferably, the encoder re-calculates the said at least one of the transcoding parameters over other parts of the P picture. The transcoding parameter is preferably the quantisation parameter.
In B pictures the encoder re-calculates at least one of the transcoding parameters over the whole of a B picture. The transcoding parameter is preferably the quantisation parameter.
In preferred embodiments DCT_type is re-used in I, P and B pictures.
Embodiments of the invention also re-use motion vectors. In an embodiment, the said transcoding parameters include, in respect of inter-encoded pictures, motion vectors which refer to parts of reference pictures, and the encoder is arranged to produce compression encoded pictures including an inter-encoded picture and a reference picture, which encoder re-uses the said preserved motion vectors for those of the said first parts of the inter-encoded picture for which the preserved motion vectors refer to parts of the reference picture which are derived.
Preferably, new motion vectors are provided for parts of the inter-encoded picture which are derived at least in part from the second video signal. More preferably, new motion vectors are provided for parts of the inter-encoded picture which are derived solely from the first signal but for which the preserved vectors refer to parts of the reference picture which are derived at least in part from the second video signal.
In embodiments, the encoder calculates motion vectors for a picture of the combined video information, and tests the calculated and preserved motion vectors against a reference map indicating, in the reference picture to which the vectors refer, the positions of information derived at least in part from the second signal.
This maintains picture quality at least for parts of the pictures which are derived only from the first signal and have motion vectors which refer to unchanged parts of a reference picture.